Telegram Group & Telegram Channel
πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:

using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes
Please open Telegram to view this post
VIEW IN TELEGRAM



tg-me.com/csharp_1001_notes/456
Create:
Last Update:

πŸ–₯ Phi-3-mini Π² 30 строках Π½Π° C# с ONNX Runtime GenAI

Π’ Ρ€Π°ΠΌΠΊΠ°Ρ… запуска Phi-3 Microsoft выпустила ΠΎΠΏΡ‚ΠΈΠΌΠΈΠ·ΠΈΡ€ΠΎΠ²Π°Π½Π½Ρ‹Π΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ ONNX, ΠΊΠ°ΠΊ ΠΏΠΎΠ΄Ρ€ΠΎΠ±Π½ΠΎ описано Π² ΡΡ‚Π°Ρ‚ΡŒΠ΅ Β«ONNX Runtime supports Phi-3 mini models across platforms and devicesΒ».
Π’Π°ΠΊΠΆΠ΅ ΠΌΠΎΠ΄Π΅Π»ΠΈ Phi-3 mini ΠΎΠΏΡƒΠ±Π»ΠΈΠΊΠΎΠ²Π°Π½Ρ‹ Π½Π° HuggingFace πŸ€—

Π˜ΡΠΏΠΎΠ»ΡŒΠ·ΡƒΡ всё это, ΠΌΠΎΠΆΠ½ΠΎ Π»Π΅Π³ΠΊΠΎ Π·Π°ΠΏΡƒΡΡ‚ΠΈΡ‚ΡŒ модСль локально всСго Π² Π½Π΅ΡΠΊΠΎΠ»ΡŒΠΊΠΈΡ… строках C#, ΠΊΠ°ΠΊ ΠΏΠΎΠΊΠ°Π·Π°Π½ΠΎ Π² этом Π³Π°ΠΉΠ΄Π΅.

▢️ Π“Π°ΠΉΠ΄

А Π²ΠΎΡ‚ Ρ‚Π΅ самыС 30 строк:


using Microsoft.ML.OnnxRuntimeGenAI;
var modelDirectory = args.Length == 2 ? args[1] :
@"C:\git\oss\Phi-3-mini-4k-instruct-onnx\cuda\cuda-int4-rtn-block-32";
using var model = new Model(modelDirectory);
using var tokenizer = new Tokenizer(model);
while (true)
{
Console.Write("Prompt: ");
var line = Console.ReadLine();
if (line == null) { continue; }

using var tokens = tokenizer.Encode(line);

using var generatorParams = new GeneratorParams(model);
generatorParams.SetSearchOption("max_length", 2048);
generatorParams.SetInputSequences(tokens);

using var generator = new Generator(model, generatorParams);

while (!generator.IsDone())
{
generator.ComputeLogits();
generator.GenerateNextToken();
var outputTokens = generator.GetSequence(0);
var newToken = outputTokens.Slice(outputTokens.Length - 1, 1);
var output = tokenizer.Decode(newToken);
Console.Write(output);
}
Console.WriteLine();
}


@csharp_1001_notes

BY C# 1001 notes





Share with your friend now:
tg-me.com/csharp_1001_notes/456

View MORE
Open in Telegram


C 1001 notes Telegram | DID YOU KNOW?

Date: |

What is Telegram?

Telegram’s stand out feature is its encryption scheme that keeps messages and media secure in transit. The scheme is known as MTProto and is based on 256-bit AES encryption, RSA encryption, and Diffie-Hellman key exchange. The result of this complicated and technical-sounding jargon? A messaging service that claims to keep your data safe.Why do we say claims? When dealing with security, you always want to leave room for scrutiny, and a few cryptography experts have criticized the system. Overall, any level of encryption is better than none, but a level of discretion should always be observed with any online connected system, even Telegram.

Telegram hopes to raise $1bn with a convertible bond private placement

The super secure UAE-based Telegram messenger service, developed by Russian-born software icon Pavel Durov, is looking to raise $1bn through a bond placement to a limited number of investors from Russia, Europe, Asia and the Middle East, the Kommersant daily reported citing unnamed sources on February 18, 2021.The issue reportedly comprises exchange bonds that could be converted into equity in the messaging service that is currently 100% owned by Durov and his brother Nikolai.Kommersant reports that the price of the conversion would be at a 10% discount to a potential IPO should it happen within five years.The minimum bond placement is said to be set at $50mn, but could be lowered to $10mn. Five-year bonds could carry an annual coupon of 7-8%.

C 1001 notes from ye


Telegram C# 1001 notes
FROM USA